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Noise reduction 



G 



The invention relates to a method and a device, in which noise filtering is 
applied. The invention further applies to a video system. 

There is presently an increasing interest in digital transmission of image 
5 sequences, e.g. through the Internet. Especially in the consumer electronics area, the sources 
of these images, such as video-cameras, video recorders, satellite receivers and others are 
affected by various types of noise. In particular, in the case of CCD and CMOS cameras, the 
sensor noise is usually modeled as white Gaussian, whereas vertical or horizontal streaks may 
be found in video scanned from motion picture films or played,by a video recorder, 

1 0 respectively. Before storage and/or transmission, it is advisable to reduce the noise level in 
the images, both to improve the visual appearance and to reduce the bit rate. Various 
algorithms are known from the art for the attenuation of noise having different distributions, 
which are generally very complex and consequently not amenable to real time 
implementation in consumer equipment, or provide poor performance, typically introducing 

1 5 artifacts and smoothing edges. 

An object of the invention is to provide less complex noise reduction. To this 
end, the invention provides a method of and a device for noise filtering and a video system as 
defined in the independent claims. Advantageous embodiments are defined in the dependent 
20 claims. 

In a first embodiment of the invention, a type of noise in the signal is 
estimated, and one of at least two noise filters is enabled, the enabled noise filter being a 
most suitable filter for the estimated type of noise. The invention is based on the insight that 
estimating a type of noise and automatically enabling one filter out of a set of simple filters, 
25 each favorable to a specific noise type, is more effective than a complex filter which has to 
cope with different noise characteristics. Both the noise type estimation and the filters have a 
low complexity and are amenable for low-cost applications. 

Edge preserving noise reduction can be achieved using spatio-temporal 
rational and median based filters. A rational filter is a filter described by a rational function, 
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e.g. the ratio of two polynomials in input variables. It is well known that spatio-temporal 
rational filters can effectively distinguish between details and homogeneous regions by 
modulating their overall low-pass behavior according to the differences of suitably chosen 
pixels [1], so that noise is significantly reduced while details are not blurred. They are 
5 effective on various types of noise, including Gaussian noise [1] and contaminated Gaussian 
noise [2]. Contaminated Gaussian noise has a probability distribution according to: 

v~(l-A)N(cr n ) + AN^) (1) 

wherein A, is a parameter and N(p) is a Gaussian distribution with variance a. A variance of 
the contaminated Gaussian distribution is given by: cr v 2 = cr n 2 (1 - X + 1 / A) (2) 

10 In case of long-tailed noise, a simple median filter [3] is used, which is effective both for 
single noisy pixels and for horizontal and vertical streaks, so that there is no need to 
distinguish between ideal and real impulsive noise. Median based operators are very efficient 
in case of long-tailed noise, especially impulsive noise, while their use in case of Gaussian 
noise is not advisable, because they tend to generate streaking and blotching artifacts. 

1 5 A further embodiment of the invention uses a simple algorithm to estimate the 

type of noise in the image sequence. This embodiment uses a kurtosis of the noise as a metric 
for the type of noise. The kurtosis is defined as [4]: 

* = // 4 /cr 4 (3) 

wherein // 4 is a fourth central moment of the data andcr is a variance of the data in the image 
20 sequence. The fourth central moment is given by // 4 = E(x-x) 4 (4) 

wherein £ is an expectation of a variable and E(x) = x . The fourth central moment // 4 is 
related to the peakedness of a single-peaked distribution. The kurtosis is dimensionless with k 
= 3 for a Gaussian distribution. A kurtosis value of 3 therefore means that the noise 
distribution has, in some sense, a same degree of peakedness as a member of the normal 
25 family. Further, k > 3 for contaminated Gaussian noise, and k » 3 for impulsive noise. 

Prior art operators which are able to distinguish among several types of noise 
are very complex. For example, in [5] a block-based, non-linear filtering technique based on 
Singular Value Decomposition that employs an efficient method of estimating noise power 
from input data is presented, however, an hypothesis of additive noise is required and only 
30 Gaussian distributions are used. In [6], in order to detect and estimate both deterministic and 
random Gaussian signals in non-Gaussian noise, the covariance of the latter is determined 
using higher order cumulants. The inverse problem is treated in [7], where signal detection 
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and classification in the presence of additive Gaussian noise is performed using higher order 
statistics. 

The input signal jc is formed by an original noise-free signal y and a noise 
signal n according to: x = y + n. In a further embodiment of the invention, the noise n is 
5 approximated by computing a difference between the signal x and the same signal being 
noise filtered, preferably in a median filter [8]. A median of N numerical values is found by 
taking a middle value in an array of the N numerical values sorted in increasing order. A 
median filter may also be referred to as a non-linear shot noise filter, which maintains high 
frequencies. Due to the well-known noise reduction and edge preserving properties of the 

10 median filter, the resulting signal, z = x -medianfx), is composed approximately of noise 
only, i.e. z = n . The kurtosis k is then estimated on z to provide an indication of the type of 
noise. Although z does not coincide with the original noise n, for reasonable values of the 
noise variance (in case of Gaussian noise or contaminated Gaussian noise) or of a percentage 
of noisy pixels (in case of impulsive noise), the parameter k allows to correctly discriminate 

15 the types of noise, using two suitable thresholds. There is no overlap in values of the 

parameter k for Gaussian, contaminated Gaussian and long-tailed noise, so that it is actually 
possible to correctly discriminate the various noise types using two thresholds, being 6 and 
15. 

Preferably, because the noise is supposed to be spatially uniform, a small part 
20 of each image (e.g. 3 by 3 pixels sub-image) is analyzed, in order to keep the computational 
load per image low. Because a stable estimate is needed, an analysis is preferably performed 
by cumulating data for a plurality of images before actually computing k. An estimate over 
900 pixels (i.e. over 100 frames) has a reasonable low variance. 

The aforementioned and other aspects of the invention will be apparent from 
25 and elucidated with reference to the embodiments described hereinafter. 



In the drawings: 

Fig. 1 shows an embodiment of a video system according to the invention; 

Figs. 2 A... 2D show exemplary spatial directions considered in the filters: Fig. 
30 2A: horizontal, Fig. 2B: vertical, Fig. 2C and Fig. 2D: diagonal; 

Fig. 3 shows an exemplary direction used by a temporal part of a rational filter 
for Gaussian noise; and 

Fig. 4 shows an exemplary combination of directions used by a temporal part 
of a rational filter for contaminated Gaussian noise. 
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The drawings only show those elements that are necessary to understand the 



invention. 



Fig. 1 shows an embodiment of a video system 1 according to the invention. 



5 The video system 1 comprises an input unit 2, such as a camera or an antenna, for obtaining 
an image sequence x. The video system 1 further comprises a noise filter 3. The noise filter 3 
comprises a noise discriminator 30 for estimating a type of noise in the image sequence x. 
The noise discriminator 30 controls a set of filters 3 1 . Depending on the estimated type of 
noise, a most suitable filter in the set of filters 31 is enabled. 
10 The noise discriminator 30 comprises a median filter 301 , a subtracter 302 and 

a noise type estimator 303. The median filter 301 filters the input signal x to obtain a filtered 
version of x, being median(x). The filtered signal median(x) is subtracted from the input 
signal x, resulting in an approximation of the noise n in the input signal x, the approximation 
given by: z = x - median(x). The signal z is furnished to the noise estimator 303 for 
15 estimating the type of noise. As described above, the estimator 303 applies a kurtosis k on the 
noise signal z. The estimator 303 furnishes a kurtosis (noise type) depending control signal to 
the set of filters 31 . Depending on the control signal from the estimator 303, one of the filters 
in the set 31 is enabled. The output >> of the noise filter 3 may be transmitted to a receiver or 
stored on a storage medium. 
20 In a preferred embodiment, the set of filters 3 1 comprises three different filters 

3 1 0, 3 1 1 , 3 12 in order to be able to treat different types of noise. Their operation is 
automatically controlled by the noise discriminator 30 as described above. Preferably, their 
support is restricted to two temporally adjacent images only, to keep the computational 
complexity low. The use of only two images has the further advantage that the amount of 
25 required image memory is lower than in methods that use more images. In this embodiment, 
the filter 3 10 is suitable for Gaussian noise, the filter 3 1 1 is suitable for contaminated 
Gaussian noise, and the filter 312 is suitable for long-tailed noise. 



3 1 1 are preferably spatio-temporal rational filters having a similar structure, constituted by 
30 the sum of a spatial and a temporal filtering part. Each filter output y 0 is computed as: 



The filters for the Gaussian noise and the contaminated Gaussian noise 310, 
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where xo, x t and x, are pixel values within a mask (xq being the central one), /, j e I describe a 
set of spatial filtering directions shown in Figs. 2A...2D, and k s and A s are suitable filter 
parameters. The temporal filtering yait,f temp has a similar form, although f temp operates also 
on pixels of a previous image, and is described below. It may be seen that the spatial filter is 

5 able to distinguish between homogeneous and detailed regions, in order to reduce noise while 
maintaining the image details. In fact, if the mask lies in a homogeneous region, the pixel 
differences (jc, - Xj) 2 which appear at the denominator are small, and the high-pass component 
present at the numerator, which is subtracted from xo, gives an overall low-pass behavior. In 
turn, if the same differences have a large value, an edge is supposed to be present, and the 

10 filter leaves the pixel unchanged in order not to blur the detail. 

The temporal part exploits the same principle of detail sensitive behavior, and 
for Gaussian noise the form is similar to that of the spatial part: 

— y p + r 

j-(gauss) _ ^ •*< ^ A 0 (7) 

tej k t \ (*, P — *o ) + A tX 

where / e J describes a set of temporal filtering directions as shown in Fig. 3. In Fig. 3 only 
1 5 one of 9 possible directions (according to the possible positions of xD has been drawn for the 
sake of clarity. The superscript p refers to pixels belonging to a previous image, and k t i and 
A t i are suitable filter parameters. 

The situation is slightly more complicated for contaminated Gaussian noise. In 
this case, details and noise are more difficult to discriminate, because the pixel noise level 
20 can be large (due to the rather long tails of the distribution), and less information with respect 
to the spatial case is available; more precisely, due to the limited temporal size of the filter 
support (only two images), pixels are available only at one (temporal) side of xo (vice- versa, 
in the spatial part of the filter 311, pixels both at the right and at the left of xq, or both on top 
of and below, are available) so that the simple denominator of the spatial part does not allow 
25 to distinguish between a single noisy pixel and the edge of an object. For contaminated 
Gaussian noise,^/^ is defined as: 

s-cont Gauss ^ i 1 ^0 /ox 

~h\k n ix? -x 0 ) 2 +k l3 (xf -x,fV2 + A l2 
where / e J describes a set of temporal filtering combinations (a combination of a temporal 
direction with a spatial direction) as shown in Fig. 4 and where ka 9 k t3 and A t 2 are suitable 
30 filter parameters. In Fig. 4 only one combination of x? and x x of a plurality of possible 
combinations has been drawn for the sake of clarity. In this case, the pixels at the 
denominator, which controls the strength of the low-pass action, are three instead of two: x /3 



(SiiHSi 3 EP " S illlHilPBi 5W^v f: lft 

6 28.02.2000 
xf and xq. In fact, as already mentioned above, it is not advisable to use the same control 
strategy as for Gaussian noise: the difference (x, p -xq) may be large due to a noise peak instead 
of an edge with consequent loss of the noise filtering action. In turn, if the same difference is 
corrected by averaging with another difference, i.e. (x, p -x,), the denominator remains low also 

5 in presence of isolated noisy pixels, and the desired low-pass behavior is obtained. 

Although the filters 3 1 0 and 3 1 1 are shown in Fig. 1 as separate filters, in a 
practical embodiment, the filters 310 and 3 1 1 are combined in one rational filter with a 
common spatial part and different temporal parts, a first temporal part for Gaussian noise and 
a second temporal part for contaminated Gaussian noise. Depending on the type of noise 

10 estimated in the noise discriminator 30, the suitable temporal part is enabled. In a further 
practical embodiment, the first temporal part and the second temporal part are implemented 
as one temporal filtering part according to equation (8), wherein in case the noise has a 
Gaussian distribution, the parameter k& is taken zero to obtain a rational filter according to 
equation (7). 

1 5 The rational filter 3 1 0/3 1 1 is enabled if the value of the kurtosis k of z is lower 

than 15, otherwise the median filter 312 is enabled. If the kurtosis k is lower than 6, the first 
temporal part (for the Gaussian noise) is enabled. If the kurtosis k is between 6 and 15, the 
second temporal part (for the contaminated Gaussian noise) is enabled. 

In order to treat long-tail noise effectively, the filter 312 is preferably a simple 
20 median filter. In general, a median filter is based on order statistics. A two-dimensional 
median filter is given by: 

y 0 = median{x t , x 0 , x y } (9) 

The set x b x y defines a neighborhood of the central pixel x 0 and is called a filter mask. The 
median filter replaces the value of the central pixel by the median of the values of the pixels 

25 in the filter mask. A simple mask, which is appropriate, is a 5 element X-shaped filter. Such a 
filter is known from [3]. In case of the 5 element X-shaped filter, the filter mask includes the 
central pixel x 0 and the pixels diagonally related to the central pixel x 0 . These spatial 
directions are indicated in Figs. 2C...D. 

Preferably, both ideal impulsive noise (single noisy pixels), and real world 

30 impulsive-like noise (e.g. present in satellite receivers) made of horizontal one pixel wide 
strips rather than by single noisy pixels, are removed. Both types of noise affect only one 
pixel out of 5 in the X-shaped mask, so that the noisy element is easily removed by the 
median operator. It is noticed, that one pixel wide vertical strips, which may be found in 
video obtained from motion picture films, can also be effectively removed by this filter. 
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To remove wider strips, a larger support is required. Once impulsive noise type has been 
detected, the simple median is used. 

The noise discriminator 30 controls the set of filters 3 1 . Although in the 
above-described embodiments, hard switching is used, soft switching is also possible, e.g. 
5 enabling the most suitable filter of the set of filters 3 1 by more than 50 % and in addition 

partly enabling one or more of the other filters in the set of filters 3 1 . In an exemplary case in 
which the signal includes mostly Gaussian noise, the filter 310 may be enabled for 80% and 
the other two filters 311 and 312 for 10%. The claims should be construed as comprising 
such a soft switching implementation too. 
10 Depending on the application or the image sequence, other filters or a different 

noise discriminator may be used. The basic idea of the invention is to use at least two filters, 
designed for different types of noise, and a noise discriminator for enabling the most suitable 
filter of the at least two filters. The invention is also applicable to other signals, e.g. audio. 

Motion-compensated based algorithms generally provide better performances 
15 at the cost of a much more complex structure. Motion-compensated based algorithms are 
preferably applied in professional embodiments of the invention. 



It should be noted that the above-mentioned embodiments illustrate rather than 
limit the invention, and that those skilled in the art will be able to design many alternative 

20 embodiments without departing from the scope of the appended claims. The word 'image' 
also refers to picture, frame, field, etc. In the claims, any reference signs placed between 
parentheses shall not be construed as limiting the claim. The word 'comprising' does not 
exclude the presence of other elements or steps than those listed in a claim. The invention can 
be implemented by means of hardware comprising several distinct elements, and by means of 

25 a suitably programmed computer. In a device claim enumerating several means, several of 
these means can be embodied by one and the same item of hardware. The mere fact that 
certain measures are recited in mutually different dependent claims does not indicate that a 
combination of these measures cannot be used to advantage. 

In summary, the invention provides noise filtering of a signal by estimating a 

30 type of noise in the signal and enabling one of at least two noise filters, the enabled noise 
filter being a most suitable filter for the estimated type of noise. An approximation of the 
noise in the signal is obtained by computing a difference between the signal and a noise- 
filtered version of the signal. The invention uses a kurtosis of the noise as a metric for 
estimating the type of noise. If the estimated type of noise is long-tailed noise, a median filter 
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is enabled to filter the signal. If the estimated type of noise is Gaussian noise or contaminated 
Gaussian noise, a spatio-temporal filter is enabled to filter the signal. The invention may be 
applied in a video system with a camera and a noise filter. 
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CLAIMS: 



1. Method of noise filtering (3) a signal (x), the method comprising the steps of: 
estimating (30) a type of noise in the signal (x); and 

enabling (30) one of at least two noise filtering operations (310,31 1,312), the 
enabled noise filtering operation (3 1 0,3 1 1 ,3 1 2) being a most suitable noise filtering operation 
5 for the estimated type of noise. 

2. Method of noise filtering (3) as claimed in claim 1, wherein: 
a median filtering operation (312) is enabled if the estimated type of noise is 

long-tailed noise; and 

10 a spatio-temporal rational filtering operation (3 1 0,3 1 1 ) is enabled if the 

estimated type of noise is Gaussian noise or contaminated Gaussian noise. 

3. Method of noise filtering (3) as claimed in claim 2, wherein the rational 
filtering operation (3 10,31 1) comprises: 

enabling a first temporal filtering operation (310) if the estimated type of noise 
is Gaussian noise; and 

enabling a second temporal filtering operation (3 1 1) if the estimated type of 
noise is contaminated Gaussian noise, 

the first temporal filtering operation (310) taking into account at least one 
temporal direction and the second temporal filtering operation (311) taking into account at 
least one combination of a temporal direction and a spatial direction. 

4. Method ofnoi.se filtering (3) as claimed in claim 1, wherein a kurtosis of the 
noise (z) is used (303) as a metric for estimating the type of noise. 

25 

5. Method of noise filtering (3) as claimed in claim 2, wherein a kurtosis of the 
noise (z) is used (303) as a metric for estimating the type of noise; 

the median filtering operation (3 12) is enabled if the kurtosis is above a first 

threshold; and 
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the rational noise filtering operation (310, 1 1) is enabled if the kurtosis is 
below said first threshold; 

6. Method of noise filtering (3) as claimed in claim 3, wherein a kurtosis of the 

5 noise (z) is used (303) as a metric for estimating the type of noise; 

the median filtering operation (312) is enabled if the kurtosis is above a first 

threshold; 

the rational noise filtering operation (3 10,3 1 1) is enabled if the kurtosis of the 
noise is below said first threshold, wherein the rational filtering operation comprises: 
10 enabling the first temporal filtering operation (310) if the kurtosis is below a 

second threshold, said second threshold being lower than said first threshold; and 

enabling the second temporal filtering operation (31 1) if the kurtosis is above 
the second threshold and below the first threshold. 

15 7. A method of noise filtering (3) as claimed in claim 6, wherein the first 

threshold is about 15 and the second threshold is about 6. 

8. Method of noise filtering (3) as claimed in claim 1, wherein the noise (z) in 
the signal is approximated by a difference (302) between the signal (x) and a noise-filtered 

20 (301) version of the signal (x). 

9. Method of noise filtering (3) as claimed in claim 8, wherein the noise-filtered 
version of the signal (x) is obtained by subjecting the signal (x) to a median filtering 
operation (301). 



25 



30 



10. Device (3) for noise filtering a signal (x), the device (3) comprising: 
means (30) for estimating a type of noise in the signal (x); and 

means (30) for enabling one of at least two noise filters (310,31 1,312), the 
enabled noise filter (310,31 1,312) being a most suitable filter for the estimated type of noise. 

1 1 . Video system (1) comprising: 

means (2) for obtaining an image sequence (x), 

a device (3) as claimed in claim 10 for noise filtering the image sequence (x). 
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ABSTRACT: 9 - . 



c 



The invention provides noise filtering (3) of a signal (x) by estimating (30) a 
type of noise in the signal (x) and enabling (30) one of at least two noise filters 
(3 1 0,3 1 1 ,3 1 2), the enabled noise filter (310,31 1,312) being a most suitable filter for the 
estimated type of noise. 

5 An approximation of the noise (z) in the signal (x) is obtained by computing 

(302) a difference between the signal (x) and a noise-filtered (301) version of the signal (x). 

The invention uses (303) a kurtosis of the noise as a metric for estimating the 
type of noise. If the estimated type of noise is long-tailed noise, a median filter (312) is 
enabled to filter the signal. If the estimated type of noise is Gaussian noise or contaminated 
10 Gaussian noise, a spatio-temporal filter (3 10,3 1 1) is enabled to filter the signal. 

The invention may be applied in a video system (1) with a camera (2) and a 

noise filter (3). 
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